Robust Resource Allocation in Heterogeneous Parallel and Distributed Computing Systems

In parallel and distributed computing multiple computers are collectively utilized to simultaneously process a set of tasks to improve performance over that of a single processor [BSB01]. Often, such computing systems are constructed from a heterogeneous mix of machines that may differ in their capabilities, e.g., available memory, number of floating point units, clock speed, and operating system. In a heterogeneous computing system, the execution time of a task may differ depending on which computer executes the task. Often, task resource requirements lead to inconsistent performance differences between heterogeneous machines. That is, machine 1 being faster than machine 2 on some task A does not imply that machine 1 is uniformly faster on all tasks. Resource allocation in heterogeneous parallel and distributed computing is the process of assigning tasks to computers for execution such that some performance objective is optimized. For example, a common objective in resource allocation is to minimize the total time required to complete a set of tasks to be executed. It has been shown that resource allocation is an NP-hard problem (e.g., [IK77]), i.e., an optimal solution cannot be found in reasonable time for problems of realistic size. Therefore, the task of resource allocation is often addressed heuristically. A resource